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Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.114. Applicant's submission filed on 4/12/2007 has been entered. 

Allowable Subject Matter 

Claims 29, 25-26, 30-31, 33-34, and 37-39 are allowed. 

The following is a statement of reasons for the indication of allowable subject matter: 
The closest prior art, Benjamin, Tuceryan, and Rong show similar limitations but not in the 
same combination of identifying a region in motion in real time further comprises estimating the 
background image bv modeling the background image as a temporal pixel average of five 
recent image frames in the real-time video imagery information. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 2, 4-5, 7-10, 12, 13, 15-17, and 20-23 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Benjamin Lok, the title is "Online model reconstruction for interactive virtual 

environments" March 2001, hereinafter Benjamin, and in view of Tuceryan et al. US 

2002/0113756 Al, (hereinafter refers as Tuceryan). 
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Claim 12. 

Benjamin in the abstract on page 69 teaches a method comprising: generating a three 
dimensional model of a three dimensional environment from range sensor information 
representing a height field (Benjamin on page 71 under section 3 right column at fourth 
paragraph teaches an elevation of an object above a table) for the environment; Benjamin on 
page 71 section 2.4 at second paragraph teaches keeping track of which source images contribute 
to a final pixel result, i.e. similar to claim limitation of" tracking orientation information of at 
least one image sensor in the environment with respect to the three dimensional model in real- 
time". Also Benjamin on page 69 under "related work" teaches in Virtual Environments required 
additional trackers to control the motion of a model. Benjamin on page 69 section 1 at first 
paragraph teaches an immersive virtual environments (IVE), see following claim limitation: 
projecting real-time video imagery information from the at least one image sensor onto the three 
dimensional model based on the tracked orientation information. Benjamin on page 70 left 
column at second paragraph teaches visualizing the three dimensional model with the projected 
real-time video imagery. Benjamin on page 73 in figures 4-5 explicitly illustrates projecting the 
real-time video imagery information comprises generating a depth map image from a video 
sensor viewpoint, and projective texture mapping the real-time video imagery information onto 
the three dimensional model conditioned upon visibility as determined from the generated depth 
map image. Examiner's note: in figs. 4-5 illustrates generating a depth map and projective 
texture mapping. Benjamin on page 69 right column at second paragraph teaches using a 
framebuffer to compute results in a massively parallel manner. 
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Benjamin does not disclose using a one-pass approach on graphics hardware that supports 
SGI OpenGL extensions, see following claim limitation: wherein generating the depth map 
image and projective texture mapping the real-time video imagery information are performed 
using a one-pass approach on graphics hardware that supports SGI OpenGL extensions. In the 
specification [0080] discloses that the approach in fig. 4 represents a two-pass process, and on 
page 32 [0080] discloses this approach can be implemented as a one-pass approach . 

However, Tuceryan teaches camera calibration methods for augmented reality. Tuceryan 
in fig. 9 illustrates tracker camera 92 and scene camera 91 are connected to SGI that supports 
OpenGL, obviously performing using a one -pass approach on graphics hardware SGI540. 

Thus, it would have been obvious to one ordinary skill in the art at the time the invention 
was made to combine Tuceryan's SGI 540 into Benjamin's SGI Reality Monster system for an 
efficient and less computationally-intensive technique to recover mapping and projecting texture 
into 3-D model. 
Claim 2. 

Benjamin on page 73 illustrates 3D models; also on page 71 right column at forth 
paragraph teaches reconstruction from different viewpoints. 
Claim 4. 

Benjamin on page 72 left column at third paragraph teaches multiple camera inputs as an 
image sensor. 
Claim 5. 

Benjamin teaches the claim limitation, because on page 71 section 2.4 teaches for each 
mesh point each vertex gets its color, and at right column third paragraph teaches the vertices are 
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modulated with the applied texture. It means there is a regular grid at a user-defined to produce 
the height and depth field. 
Claim 7. 

Benjamin in the abstract teaches generating models with lighting in IVEs, and gives 
results using synthetic and real data. Benjamin uses a novel visual hull. 
Claims 8-9. 

Benjamin on page 70 left column first and second paragraphs teaches the claim 
limitations as the system enables users to see their arm and a real book in the VE, and naturally 
reach out and pick up the book. The system has two distinct goals, providing the user visual 
feedback of his body and nearby real objects, and generating real time models for rendering, 
simulations, and interactions with the virtual environment. The cameras can be situated remotely 
to have a global navigational satellite system receiver, as Benjamin on page 72 section 4 teaches 
the user is tracked with a scalable wide-area traker. 
Claim 10. 

Benjamin in fig. 2 illustrates multiple image sensors as multi cameras and projecting the 
real time imagery information onto 3D model. 
Claim 13. 

Benjamin does not teach using a stereo video-projector coupling to a user's head position. 
However, Tuceryan in fig. 9 illustrates the claim limitations. 

Thus, it would have been obvious to one ordinary skill in the art at the time the invention 
was made to combine Tuceryan's SGI 540 into Benjamin's SGI Reality Monster system for an 
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efficient and less computationally-intensive technique to recover mapping and projecting texture 
into 3-D model. 
Claim 22. 

Benjamin in the abstract on page 69 teaches a system comprising: generating a three 
dimensional model of a three dimensional environment from range sensor information 
representing a height field (Benjamin on page 71 under section 3 right column at fourth 
paragraph teaches an elevation of an object above a table) for the environment; Benjamin on 
page 71 section 2.4 at second paragraph teaches keeping track of which source images contribute 
to a final pixel result, i.e. similar to claim limitation of" tracking orientation information of at 
least one image sensor in the environment with respect to the three dimensional model in real- 
time". Also Benjamin on page 69 under "related work" teaches in Virtual Environments required 
additional trackers to control the motion of a model. Benjamin on page 69 section 1 at first 
paragraph teaches an immersive virtual environments (IVE), see following claim limitation: 
projecting real-time video imagery information from the at least one image sensor onto the three 
dimensional model based on the tracked orientation information. Benjamin on page 70 left 
column at second paragraph teaches visualizing the three dimensional model with the projected 
real-time video imagery. Benjamin on page 73 in figures 4-5 explicitly illustrates projecting the 
real-time video imagery information comprises generating a depth map image from a video 
sensor viewpoint, and projective texture mapping the real-time video imagery information onto 
the three dimensional model conditioned upon visibility as determined from the generated depth 
map image. Examiner's note: in figs. 4-5 illustrates generating a depth map and projective 
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texture mapping. Benjamin on page 69 right column at second paragraph teaches using a 
framebuffer to compute results in a massively parallel manner. 

Benjamin does not disclose using a one-pass approach on graphics hardware that supports 
SGI OpenGL extensions, see following claim limitation: wherein generating the depth map 
image and projective texture mapping the real-time video imagery information are performed 
using a one-pass approach on graphics hardware that supports SGI OpenGL extensions. In the 
specification [0080] discloses that the approach in fig. 4 represents a two-pass process, and on 
page 32 [0080] discloses this approach can be implemented as a one-pass approach . 

However, Tuceryan teaches camera calibration methods for augmented reality. Tuceryan 
in fig. 9 illustrates tracker camera 92 and scene camera 91 are connected to SGI that supports 
OpenGL, obviously performing using a one-pass approach on graphics hardware SGI540. 
Thus, it would have been obvious to one ordinary skill in the art at the time the invention was 
made to combine Tuceryan' s SGI 540 into Benjamin's SGI Reality Monster system for an 
efficient and less computationally-intensive technique to recover mapping and projecting texture 
into 3-D model. 
Claim 15. 

Benjamin in fig. 2 illustrates multiple image sensors as multi cameras and projecting the 
real time imagery information onto 3D model. 
Claim 16. 

The claim limitation is obvious because Benjamin in the abstract teaches real objects in 
an immersive virtual environment for visualization and interaction. Also on page 71 section 4 at 
third paragraph teaches video and memory. 
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Claim 17. 

The claim limitation is obvious because when generating a 3D image, the viewpoint of 
the video imagery projection should be separate from viewpoints from multiple image sensors 
(see, Benjamin's multi cameras in fig. 2). 
Claim 20. 

Benjamin in fig. 2 illustrates multiple image sensors as multi cameras and projecting the 
real time imagery information onto 3D model. 
Claim 21. 

Benjamin on page 70 left column first and second paragraphs teaches the claim 
limitations as the system enables users to see their arm and a real book in the VE, and naturally 
reach out and pick up the book. The system has two distinct goals, providing the user visual 
feedback of his body and nearby real objects, and generating real time models for rendering, 
simulations, and interactions with the virtual environment. The cameras can be situated remotely 
to have a global navigational satellite system receiver, as Benjamin on page 72 section 4 teaches 
the user is tracked with a scalable wide-area traker. 
Claim 23. 

Benjamin does not teach using a stereo video-projector coupling to a user's head position. 
However, Tuceryan in fig. 9 illustrates the claim limitations. 

Thus, it would have been obvious to one ordinary skill in the art at the time the invention 
was made to combine Tuceryan's SGI 540 into Benjamin's SGI Reality Monster system for an 
efficient and less computationally-intensive technique to recover mapping and projecting texture 
into 3-D model. 
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Claim 3, 6, 18-19 and 45-47 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Benjamin Lok, the title is "Online model reconstruction for interactive 
virtual environments" March 2001, and in view of Tuceryan et ah US 2002/0113756 Al, 
and further in view of Rong et al. US 6,879,946 B2, hereinafter Rong, 

Claim 3. 

Benjamin and Tuceryan do not teach selecting geometric primitives from a group 
including a sphere primitive and a cuboid primitive. 

However, Rong at col. 2 lines 50-57 teaches the claim limitation. Rong at col. 3 line 6 
teaches fitting complex 3D shapes. 

Thus, it would have been obvious to one ordinary skill in the art at the time the invention 
was made to combine Rong's AHR (Attributed hypergraph representations) into Tuceryan and 
Benjamin in order to improve modeling of a 3D scene and by applying a set of AH operators to 
its AHR. The AHR method forms a data compression system for efficient web streaming over 
the Internet. 
Claim 6. 

Benjamin and Tuceryan do not teach hole fitting and tessellation. 

However, Rong at col. 13 lines 34-38 teaches hole filling (by evaluating the distance 
between pairs of neighboring points) and tessellation (see col. 3 line 50) to generate a polygon 
mesh. 

Thus, it would have been obvious to one ordinary skill in the art at the time the invention 
was made to combine Rong's AHR (Attributed hypergraph representations) into Tuceryan and 
Benjamin in order to improve modeling of a 3D scene and by applying a set of AH operators to 
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its AHR. The AHR method forms a data compression system for efficient web streaming over 
the Internet. 
Claim 18. 

Benjamin on page 73 illustrates 3D models; also on page 71 right column at forth 
paragraph teaches reconstruction from different viewpoints (i.e. similar to different sections of 
the structure, in the claim). Benjamin and Tuceryan do not teach selecting geometric primitives 
based at least in part on input from a person regarding different shapes. 

However, Rong at col. 2 lines 50-57 teaches selecting geometric primitives based at least 
in part on input from a person regarding different shapes. Rong in the abstract teaches a range 
information is first computed then a model is constructed. Rong at col. 3 line 6 teaches fitting 
complex 3D shapes. 

Thus, it would have been obvious to one ordinary skill in the art at the time the invention 
was made to combine Rong's AHR (Attributed hypergraph representations) into Tuceryan and 
Benjamin in order to improve modeling of a 3D scene and by applying a set of AH operators to 
its AHR. The AHR method forms a data compression system for efficient web streaming over 
the Internet. 
Claim 19. 

Benjamin teaches the claim limitation, because on page 71 section 2.4 teaches for each 
mesh point each vertex gets its color, and at right column third paragraph teaches the vertices are 
modulated with the applied texture. It means there is a regular grid at a user-defined to produce 
the height and depth field. 

Benjamin and Tuceryan do not teach hole fitting and tessellation. 
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However, Rong at col. 13 lines 34-38 teaches hole filling (by evaluating the distance 
between pairs of neighboring points) and tessellation (see col. 3 line 50) to generate a polygon 
mesh. 

Thus, it would have been obvious to one ordinary skill in the art at the time the invention 
was made to combine Rong's AHR (Attributed hypergraph representations) into Tuceryan and 
Benjamin in order to improve modeling of a 3D scene and by applying a set of AH operators to 
its AHR. The AHR method forms a data compression system for efficient web streaming over 
the Internet. 
Claim 45. 

Benjamin and Tuceryan do not teach selecting geometric primitives from a group 
including a sphere primitive and a cuboid primitive. 

However, Rong at col. 2 lines 50-57 teaches the claim limitation. Rong at col. 3 line 6 
teaches fitting complex 3D shapes. Rong at col. 2 lines 35-29 teaches that the geometrical 
modeling is the basic requirement for any vision or graphics system. 

Thus, it would have been obvious to one ordinary skill in the art at the time the invention 
was made to combine Rong's AHR (Attributed hypergraph representations) into Tuceryan and 
Benjamin in order to improve modeling of a 3D scene and by applying a set of AH operators to 
its AHR. The AHR method forms a data compression system for efficient web streaming over 
the Internet. 
Claim 46. 

Benjamin and Tuceryan do not teach selecting geometric primitives from a group 
including a sphere primitive and a cuboid primitive. 
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However, Rong at col. 2 lines 50-57 teaches the claim limitation. Rong at col. 3 line 6 
teaches fitting complex 3D shapes. Rong at col. 2 lines 35-29 teaches that the geometrical 
modeling is the basic requirement for any vision or graphics system. 

Thus, it would have been obvious to one ordinary skill in the art at the time the invention 
was made to combine Rong's AHR (Attributed hypergraph representations) into Tuceryan and 
Benjamin in order to improve modeling of a 3D scene and by applying a set of AH operators to 
its AHR. The AHR method forms a data compression system for efficient web streaming over 
the Internet. 
Claim 47. 

Benjamin and Tuceryan do not teach selecting geometric primitives from a group 
including a sphere primitive and a cuboid primitive. 

However, Rong at col. 2 lines 50-57 teaches the claim limitation. Rong at col. 3 line 6 
teaches fitting complex 3D shapes. Rong at col. 2 lines 35-29 teaches that the geometrical 
modeling is the basic requirement for any vision or graphics system. 

Thus, it would have been obvious to one ordinary skill in the art at the time the invention was 
made to combine Rong's AHR (Attributed hypergraph representations) into Tuceryan and 
Benjamin in order to improve modeling of a 3D scene and by applying a set of AH operators to 
its AHR. The AHR method forms a data compression system for efficient web streaming over 
the Internet. 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Javid A. Amini whose telephone number is 571-272-7654. The 
examiner can normally be reached on 8-4pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee Tung can be reached on 571-272-7794. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Javid A Amini 
Examiner 
Art Unit 2628 



J.A. 




